Alphabet's Loon has been using algorithmic processes to optimize the flight of its stratospheric balloons for years now -- and setting records for time spent aloft as a result. But the company is now deploying a new navigation system that has the potential to be much better, and it's using true reinforcement-learning AI to teach itself to optimize navigation better than humans ever could. Loon developed the new reinforcement-learning system, which it says is the first to be used in an actual product aerospace context, with its Alphabet colleagues at Google AI in Montreal over the past couple of years. Unlike its past algorithmic navigation software, this one is devised entirely by machine -- a machine that's able to calculate the optimal navigation path for the balloons much more quickly than the human-made system could, and with much more efficiency, meaning the balloons use much less power to travel the same or greater distances than before. How does Loon know it's better?

Efficiently navigating a superpressure balloon in the stratosphere1 requires the integration of a multitude of cues, such as wind speed and solar elevation, and the process is complicated by forecast errors and sparse wind measurements. Coupled with the need to make decisions in real time, these factors rule out the use of conventional control techniques2,3. Here we describe the use of reinforcement learning4,5 to create a high-performing flight controller. Our algorithm uses data augmentation6,7 and a self-correcting design to overcome the key technical challenge of reinforcement learning from imperfect data, which has proved to be a major obstacle to its application to physical systems8. We deployed our controller to station Loon superpressure balloons at multiple locations across the globe, including a 39-day controlled experiment over the Pacific Ocean.

Project Loon is using balloons such as this to set up an aerial wireless network for telecommunications.Credit: Loon The goal of an autonomous machine is to achieve an objective by making decisions while negotiating a dynamic environment. Given complete knowledge of a system's current state, artificial intelligence and machine learning can excel at this, and even outperform humans at certain tasks -- for example, when playing arcade and turn-based board games1. But beyond the idealized world of games, real-world deployment of automated machines is hampered by environments that can be noisy and chaotic, and which are not adequately observed. The difficulty of devising long-term strategies from incomplete data can also hinder the operation of independent AI agents in real-world challenges. Writing in Nature, Bellemare et al.2 describe a way forward by demonstrating that stratospheric balloons, guided by AI, can pursue a long-term strategy for positioning themselves about a location on the Equator, even when precise knowledge of buffeting winds is not known.